Evaluation of usability and user feedback to guide telepharmacy application development in Indonesia: a mixed-methods study

Background In Indonesia, the adoption of telepharmacy was propelled by the COVID-19 pandemic, prompting the need for a user-friendly application to support both the general population and pharmacists in accessing healthcare services. Therefore, this study aimed to evaluate usability and user feedback of a pioneering telepharmacy application known as Tanya Obat (translating to “Ask about Medications”) in Indonesia, from the perspectives of the general population and pharmacists. Methods A mixed-methods sequential study was conducted with the early-stage Tanya Obat application in Bandung City. Participants, including the general population and pharmacists, were instructed to use the application for a week. Questionnaires for the general population and pharmacists were distributed from March to May and February to June 2023, respectively. The System Usability Scale questionnaire was adopted to describe usability of the developed application. Further exploration of the quantitative results required collecting open-ended feedback to assess the impressions of the participants, difficulties encountered, and desired features for enhanced user-friendliness. The collected statements were summarized and clustered using thematic analysis. Subsequently, the association between the characteristics of participants and perceived usability was determined with the Chi-square test. Result A total of 176 participants, comprising 100 individuals from the general population and 76 pharmacists, engaged in this study. In terms of usability, the questionnaire showed that Tanya Obat application was on the borderline of acceptability, with mean scores of 63.4 and 64.1 from the general population and pharmacists, respectively. Additionally, open-ended feedback targeted at achieving a more compelling user experience was categorized into two themes, including concerns regarding the functionality of certain features and recommendations for improved visual aesthetics and bug fixes. No significant associations were observed between the characteristics of participants and perceived usability (p-value > 0.05). Conclusion The results showed that the perceived usability of Tanya Obat developed for telepharmacy was below average. Therefore, feature optimizations should be performed to facilitate usability of this application in Indonesia. Supplementary Information The online version contains supplementary material available at 10.1186/s12911-024-02494-3.


Background
The development of telepharmacy, with technology incorporation into pharmaceutical care services, was propelled by the COVID-19 pandemic.Telepharmacy provides remote counseling, medication information, online purchases, adverse effects monitoring, and therapy tracking through digital platforms [1,2].Moreover, it creates a virtual channel for pharmaceutical care, connecting pharmacists and patients remotely to facilitate easy health evaluation [3].Other benefits include reducing direct interaction between healthcare professionals and patients, improving pharmaceutical service quality, and minimizing medication errors and adverse effects through various channels such as application, text messaging, video, and voice calls [2].Telepharmacy improves cost-effectiveness, healthcare access, and after-hours availability, as well as reduces travel time to healthcare facilities [4,5], breaking geographical barriers and enhancing healthcare accessibility [6,7].However, implementing this system in clinical practice may be challenging due to legal considerations, operational costs, and patient trust-building [8,9].
Recent guidelines in Indonesia mainly address general aspects of delivering telehealth services during the COVID-19 pandemic, particularly focusing on definitions and procedures [10].Specific regulations for telemedicine are necessary to ensure compliance with ethics and other regulations [11].Additionally, evaluation of usability and applicability is crucial for optimal implementation and adoption in real-world settings.Gaining insights into usability from the perspectives of the general population and pharmacists during the design and testing stages can enhance application effectiveness, which is essential for successful implementation and scalability [12][13][14].Despite the rapid growth of telepharmacy in the country [15][16][17][18], only a few available application is tested [19][20][21], leading to varied usability experiences based on features and limitations encountered [19][20][21].Many types of Indonesian telepharmacy application lack appropriate study designs and methods to address engagement issues faced by users and pharmacists.Considering the interconnection between program usability and the engagement of users, usability studies are essential for understanding and improving engagement [22].Usability is defined as a user interface characteristic that facilitates application adoption, effectiveness, efficiency, and satisfaction in a targeted environment [23].In telepharmacy implementation, poor usability can hinder technology acceptance [24].Therefore, the products, systems, processes, and procedures constituting telepharmacy must be designed and implemented to be usable, useful [25], accessible, and user-friendly for both healthcare providers and patients [26].Engaging these users in usability testing can help to address specific needs and preferences, promoting successful technology acceptance and adoption [27].
An Indonesian telepharmacy application known as Tanya Obat, translating to "Ask about Medication", is developed through design-based study [28] by a team of pharmacists and academicians.This provides a comprehensive ecosystem connecting pharmacies, pharmacists, and technicians with the general population.Furthermore, the application comprises features such as locating nearby pharmacies, online medication purchasing, consultation services, medication use information, and educational resources including webinars, e-modules, and coaching clinics for pharmacists and technicians.Tanya Obat differs from other types of application in Indonesia by offering a dedicated ecosystem for pharmacists, including online medication consultation and educational opportunities to enhance competency in delivering health services.Therefore, this study aimed to evaluate Tanya Obat usability and obtain user feedback on the application at early development stage in Bandung City from the perspectives of the general population and pharmacists.The association between the characteristics of participants and perceived usability will be explored.

Study design
A mixed-methods sequential study design was used to assess usability and obtain user feedback on Tanya Obat application at the early development stage before launching.Furthermore, questionnaires for the general population and pharmacists were distributed from March to May and February to June 2023, respectively.Approval with contract number 670/UN6.KEP/EC/2022 was received from the Research Ethics Commission of Universitas Padjadjaran, Indonesia, and all recruited participants provided written informed consent.

Study population and settings
Participants recruited in Bandung City comprised untrained and first-time Tanya Obat users from the general population, irrespective of being health system users, as well as pharmacists registered in the application or not.One week was provided for Tanya Obat usage without supervision to permit full exploration of the application, and recruiting was performed online through a convenience sampling method.Inclusion criteria for the general population consisted of individuals with (i) age above 18 years, (ii) ability to use a smartphone, (iii) lack of prior application experience, and (iv) willingness to participate.Meanwhile, the inclusion criteria for pharmacists were (i) current employment at a Community Health Center (CHC), hospital, or pharmacy, (ii) ability to use a smartphone, (iii) no prior experience with the tested application, and (iv) willingness to participate.All participants received an explanation regarding the study stages, application installation, and features.

Tanya Obat application
Tanya Obat application was developed by a team of pharmacists and pharmacy academicians in collaboration with software engineers using Dart (https://dart.dev/)programming language.As shown in Fig. 1, the application offers five main features including:

Finding Nearby Pharmacies
This feature allows users to conveniently search for the price sites of nearby pharmacies and view available medication lists, saving time spent on locating the physical buildings.

Medication Purchasing
The medication purchasing feature extends the reach of pharmacies to patients through a digital platform.Users can freely browse categories of non-prescription and prescription medications which can be ordered with the delivery service available on the application.

Ask a Pharmacist
This flagship feature enables digital consultation with pharmacists regarding self-medication, regular medications currently in use, and other concerns about consumption and side effects.Additionally, it offers pharmacists the opportunity to gain consultation experience, which can subsequently be converted into a credit unit for pursuing professional competency certification.Recruiting for this feature was performed by distributing information through webinars, scientific events, social media and ads, personal and group chat application, as well as direct visits to pharmacies.Users will be charged IDR 20,000 or USD 1.5 for 15-minute consultations based on regulations established by the Indonesian Pharmacists Association in 2019.

Health Article
Tanya Obat application allows both the general population and pharmacists to access accurate medication information in a clear and easily understandable manner.Moreover, the authors of this study were accredited pharmacists, pharmacy graduates, and students with competencies and excellent knowledge in writing health articles.Each published article was successfully curated and reviewed by professionals and an editorial team.

Sustainable Educational Development for Pharmacists
This feature can only be accessed by pharmacists, serving as a channel for enhancing educational knowledge and professional competency through webinars, e-modules, and coaching clinics.Each feature of Tanya Obat application is available for both the general population and pharmacists.However, "Ask a Pharmacist" has slightly different user interfaces tailored for the respective group of participants.This specific feature shows a list of online pharmacists for the general population.Meanwhile, the inability of accounts designed for pharmacists to request consultations as patients, is complemented by the provision of access to current and completed consultation history.

Measurements
Perceived usability was evaluated based on a standardized quantitative System Usability Scale (SUS).Meanwhile, additional feedback from users engaged in this study was assessed through open-ended qualitative questions.

Usability measurement
Usability, often defined as user interface characteristic that facilitates application adoption, effectiveness, efficiency, and satisfaction in a targeted environment [23], was assessed with the Indonesian version of the standardized SUS questionnaire [29,30].This evaluation was conducted by identifying potential user interface, functionality, and design issues based on the perspective of untrained, first-time users from the general population and pharmacists.Moreover, SUS questionnaire comprised 10 questions rated on a 5-point Likert scale ranging from [1] "Strongly Disagree", "Disagree", "Neutral", "Agree", to "Strongly Agree" [5].Participants responded based on the subjective assessment of usability, as presented in Table S1 Supplementary data.Each question contributed a score, with different scoring conversions for odd [1,3,5,7,9] and even [2,4,6,8,10] questions, reflecting positive and negative responses, respectively.Summation of all the points yielded a maximum score of 40 which generated a scale ranging from 0 to 100 through multiplication by 2.5.Higher scores suggested favorable user perceptions of the application, while lower scores signified low usability [31].A previous study showed a significant association between SUS scores reported as continuous or dichotomous data and outcomes [32].Reporting as dichotomous data may be more practical in real-world scenarios due to the ease of interpretation [33].Therefore, in this study, a SUS score exceeding 68 points was considered above average in perceived usability [34], calculated based on a published formula [33].
The questionnaire was tested for construct validity and internal consistency using 30 participants who were not included in the main analyses.Construct validity was assessed with the Pearson Product Moment correlation by correlating the score of each question with the total score [35].The questionnaire was deemed valid when the correlation coefficient (r-value) exceeded the critical value [28].The values for each question were > 0.361, signifying correlation with the respective dimension and affirming this construct as a measurement instrument.Additionally, Cronbach's Alpha coefficient yielded a score of 0.721, meeting the criterion established for internal consistency, namely a value > 0.60 [36].

Additional user feedback
After completing SUS questionnaire, participants provided additional feedback based on open-ended questions to obtain impressions following one week of Tanya Obat usage.Three questions were used to assess the perceptions of participants about the application (e.g.,), potential confusion or difficulties encountered (e.g.,), and desired features for enhanced user-friendliness.These included "What was your first impression of this application?","What would you say was the most challenging aspect of using this application?",and "What are your recommendations to improve the application?".Subsequently, a summary of interpretations made by the study team was sent to all participants for verification.

Demographic characteristics
The survey collected sociodemographic information about the participants, including factors such as gender (male, female), age, highest education level (junior high school, senior high school, bachelor's degree, registered pharmacists, master's degree, doctorate), practice settings, and years of practice (for pharmacists).Additionally, participants provided information on subjective experiences with smartphone application (amateur, beginner, skilled, highly skilled), daily duration of smartphone usage in hours (< 1, 1−2, 3−4, and > 4), and internet accessibility at home (available, unavailable).

Sample size calculation
The sample size was calculated using Slovin's formula [37] due to a lack of prior knowledge about outcome distributions.To achieve a 95% confidence interval (CI) and a margin error of 0.15 with a statistical power of 80%, a minimum sample size of 100 for the general population and 64 pharmacists was required.

Data collection
The link to both the application and survey was disseminated through a digital leaflet across social media, as well as online personal and group chat platforms.This application was available for download on the Google Play store, and data were collected using the Qualtrics platform.To prevent duplicate entries, participants were restricted to filling out the questionnaire only once from a single email address.For pharmacists, a convenient sample was formally or personally recruited through the Indonesian Pharmacists Association network.The estimated time for questionnaire completion was 10 min, while participants were initially reminded that the study aimed to assess user-friendliness of the application and not the ability to use the device correctly.Following one week of exploring the application, participants were immediately instructed to complete the questionnaire to avoid bias introduced by repeated usage and becoming overly familiar with the application.Additionally, screenshots of the features found were requested as evidence of exploring the application.

Data analysis
Basic demographic characteristics were examined using descriptive statistics, where mean SUS scores were analyzed for each subgroup of the general population and pharmacists.Survey quantitative data were tabulated and presented as frequencies, then a Chi-square test was used to determine the association between the characteristics of participants and perceived usability.Open-ended questions were summarized and clustered with thematic analysis, following steps including familiarization with data, initial code generation, theme development, theme review, theme definition, and conduction of independent qualitative analysis by JAS and DQA using NVivo software version 12. Any disagreements among JAS and DQA were resolved through discussion with a third study team member (SDA).To ensure content analysis reliability, continuous discussion and negotiation regarding the content of keywords, broader concepts, and units of meaning were performed among the team.

Demographic characteristics
A total of 176 participants (100 individuals from the general population and 76 pharmacists) engaged in this study.Those from the general population were recruited from 26 districts in Bandung City, majorly comprising 18−25 years old (59%) females (71%) with high school qualifications (71%).Furthermore, most participants were 'skilled' at using smartphone application (56%), with over four hours of daily smartphone usage (65%), and found to use Wi-Fi at home (72%), as shown in Table 1.
Subsequently, a total of 76 responses were obtained from pharmacists in Bandung City, Indonesia.These majorly included females (73.7%), working in community pharmacies (79.0%), with over six years of job experience (38.2%), considered 'skilled' in using mobile application (64.5%), possessing access to the internet (89%), and known to use the smartphone for more than four hours daily (63.2%).

Associations between the characteristics of participants and perceived usability
The characteristics of the general population and pharmacists were presented in Tables 2 and 3, respectively.This study identified no significant correlations between the characteristics and perceived application usability (p-value > 0.05).

Usability results
According to the calculation matrix in Table S2 Supplementary data, the average SUS score of the general population was 63.4, representing a below-average perceived usability.A significant variation in scores was observed, ranging from a high of 87.5 to a low of 30.The average SUS score of pharmacists was 64.1 (Table S3, Supplementary data), which was also below average, with a significant variation in scores ranging from 95 to 42.5.

Qualitative results
Two themes, including concerns and recommendations, were identified from the responses provided to the openended questions.These were further divided into subthemes comprising feature functionality for concerns as well as visual enhancements, feature improvements, system improvements, and other parameters for recommendations, as presented in Table 4 along with examples of supporting quotes.

User feedback: the general population
Concerns identified for Tanya Obat were features such as consulting a pharmacist, searching for online medication purchases, and locating nearby pharmacies.Users reported that the application was not functioning optimally and required assistance using features including purchasing medication, redeeming prescribed medicines, and consulting a pharmacist.Additionally, inconsistencies in color schemes and delays or errors when accessing features within the application were noted (Table 4).Potential enhancements for the application included creating more visually appealing shows in terms of color combinations, layout, and font styles.Additionally, it was suggested to ensure that features function as intended, providing relevant descriptions for each feature, and enhancing filters.System-related recommendations comprised addressing bugs or errors such as loading failures, adding a user guide, improving maps, and enhancing search speed.Other suggestions included introducing Tanya Obat on iOS devices and promoting the application among a wider demographic (Table 4).

User feedback: pharmacists
Concerns observed in using Tanya Obat application were related to medication purchase, address configuration, and specific hurdles in the registration of pharmacists.
Another concern was identified with the "consult a pharmacist" feature, where the need for more assistance was reported (Table 5).
Several suggestions obtained from feedback in openended questions were majorly related to features.Regarding the medication purchase feature, the requirement for expanded stock availability and the incorporation of new functionalities was reiterated.Furthermore, pharmacists stated the need for an online Continuing Professional Development (CPD) feature, which would be incorporated by the Indonesian Pharmacists Association as the issuer of competency certificates.This suggestion was provided to support consistent education through seminars, articles, and learning modules, along with the integration of counseling history or medical records of patients that can be accessed by pharmacists in Indonesia and claimed as credit points for CPD.The feature for registration of pharmacists should be enhanced, while modification of the color palette was recommended to ensure increased visibility and maintain satisfactory show and font choices.

Principal observations
In this study, usability and user feedback of the innovative Tanya Obat application were evaluated.Perceived "The screen needs to be improved as it does not have a good contrast and lay out given resolution and size." (Participant 15) Modify the logo and color scheme of the application to be more appealing.
"I think the logo and its color need to be improved to be more appealing to users." (Participant 13) Redesign the homepage layout and make illustrations in notifications more informative or interactive to prevent monotony.
"Would be better to re-design the lay out and make notifications more interactive.Now it seems monotone." (Participant 22) Opt for darker colors when using white text.
"Please use darker color when using white text.White text with bright color makes it difficult to read." (Participants 10) Implement font size adjustments for both features and articles, considering that article fonts may be too small for some readers.
"Health articles were difficult to read because the font size is too small.I am sure others can't read as well.Moreover, the small font size in another feature needs to be adjusted." (Participant 9) Adjust font colors for readability and ensure consistency in font style and color."In order to simplify the search option, please enhance the filter options, for example, city where you live, neighborhood, district, opening hours as well as user testimony using star (ratings)." (Participant 32) Include informative notes, such as acceptance of national health insurance or information about adjacent clinics."It would be nice if important detail such as whether the pharmacies accept national health insurance or more info regarding adjacent clinics can be added to the application." (Participant 19) Medication purchase 45 (45) Include detailed information about indications, composition, dosage, side effects, suitability for specific demographics, images, alternatives, pricing, shipment origin, and estimated delivery time."Users will benefit from detailed info about medication, such as indications, composition, dosage, side effects, suitability for specific demographics, images, alternatives, pricing, origin of shipment, and estimated delivery time." (Participant 24) Incorporate a search feature for recommended medicines based on ailments, along with alternatives.
"I suggest that the application has a feature to look for a recommended medicine based on symptoms or ailments, also with their alternatives." (Participant 17) Integrate filters to compare prices from lowest to highest, and distinguish between generic and patented drugs.
"The filters feature needs to be improved, for example something to compare prices of medicine from lowest to highest, distinguish between generic and patented drugs,…." (Participant 35) Add a reminder alarm for medication consumption and recommendations for alternative and complementary medicine.
"Since this is application about medication, I will appreciate it if it also has a reminder alarm for me to take my routine medication.Moreover, feature about recommendations for alternative and complementary medicine would be beneficial for users" (Participant 87) Table 4 General population concerns and recommendations for Tanya Obat application (N = 100) usability was categorized as below average, with average scores of 63.4 and 64.1 for the general population and pharmacists, respectively.The identified concerns primarily included the functionality of some features, while chances for improvement were observed in the areas of visual, features, and system enhancements.These concerns and recommendations slightly differed between the general population and pharmacists.
From the general population perspective, the belowaverage score obtained might be attributed to disparities in digital literacy, defined as the ability to acknowledge and use information from various sources presented through computers [38].A national survey conducted among 10,000 Indonesians showed a digital literacy index of 3.49 in 2021 and 3.54 on a scale of 5 in 2022 [39][40][41], and despite the improvement, digital literacy in the country remained at a moderate level.Previous results showed that the general population and pharmacy students in Indonesia had a positive perception and were willing to use telepharmacy services [42,43], providing an opportunity for successful implementation of the application.
Several areas for improvement were identified to enhance and adapt the future application to the understanding and capabilities of users, thereby leading to a more usable and useful system [25 ].Moreover, primarily existing problems included complications using the features, attributed to unfamiliarity with technology, lack of confidence in using electronic devices, or fear of committing mistakes [39].A survey reported that 69% of 10,000 Indonesians have not accessed health services through digital platforms [39].Complications while running the application can be reduced by simplifying the log-in process, reducing required tasks, and showing fewer buttons on the screen [44][45][46][47][48][49][50][51][52][53][54][55].Selecting the appropriate design, wording, and development language in a user-centered and participatory design process is crucial and may have an important impact on engagement [56].These improvements tend to increase satisfaction, which plays a vital role in the implementation and continual use of the application [57,58].Therefore, developers must prioritize maximizing the application performance to be more user-friendly [44,[59][60][61][62][63][64].
The similar marginally acceptable score obtained from the feedback of pharmacists signified that more work was needed for usability improvement, as these professionals were expected to be more exposed to telepharmacy application than the general population.This unsatisfying score could be partly explained by the varying levels of readiness to use telepharmacy application [65].Pharmacists were forced to implement telehealth services without adequate readiness assessment, particularly in a setting without well-established telehealth services before the COVID-19 pandemic, such as Indonesia [66] Furthermore, the main concerns that require adjustment to enhance application usability and reduce errors include expanding the network of available pharmacies and pharmacists with specialties comprising traditional medication and cosmetics.Specialization among pharmacists has various important benefits such as higher adherence and persistence, better clinical outcomes, monetary benefits for both patients and the healthcare system, and higher patient satisfaction [67].Therefore, expanding the network of the pharmacy sector to include

No. Category Concerns
Frequency (N (%)) Example Quote 3 System Improvements 32 (32) Address recurring bugs or errors, particularly issues with loading failures for purchasing medicine, locating nearby pharmacies, and consulting a pharmacist.
"The errors need to be immediately solved, for example with loading failures, since it will increase user' satisfaction." (Participant 77) Provide a guide for first-time users upon login.
"The easy-to-understand guide is needed to be read before using the application." (Participant 56) Enhance the mapping system to prevent errors such as incorrect medication deliveries."The mapping system needs to be very accurate, in order to avoid error in delivering the medication.traditional medication specialists and pharmacist cosmetologists will significantly improve the application.
Recommendations of pharmacists regarding giving details about all medication were consistent with a previous study that reported an increment of adherence among patients after providing simple and brief written medication information [68].Additionally, offering live interaction during counseling led to care quality improvement.Specific concerns need to be addressed, such as ensuring properly configured camera and audio settings, appropriate light quality, stable internet connection, and readiness to assist patients unfamiliar with the technology [69].Furthermore, pharmacists recommended interactive video learning for application enhancement.This was consistent with another study that showed effectiveness in interactive video [70,71], with an 89.7% reported increase in learning outcomes [70].Interactive virtual content had significant effectiveness compared to the online class group method [71].Another recommendation was related to visualization enhancement, playing an important role in attracting user attention [44] with a tendency to influence perceived usability even when no differences existed in functionality offered [72].Users preferred simplicity, showing more graphics than crowded text, and consistent style using combinations of colors [45][46][47][73][74][75].Consequently, telepharmacy application usability would be optimized with the results of this study before proceeding to evaluate effectiveness because the standards of health professionals need to be met [76].The characteristics of the general population and pharmacists were found to have no significant association with perceived usability.These results showed that the application could be used for all age groups irrespective of educational background, familiarity with smartphone operation, and internet accessibility level [77].However, certain studies reported inconsistent results on the association between sociodemographic factors and perceived usability among the general population and pharmacists, respectively [77][78][79].Significant associations were previously shown between demographic-related variables and usability [80] from an investigation conducted among participants with years of mobile application experience [80].The less experienced participants using mobile health application in Indonesia and their subjectivity might lead to under or overestimating the experience had with smartphones, which could clarify the non-significant associations observed in this study [81].Moreover, some unmeasured factors tended to be associated with perceived usability, such as readiness [81], experience in using mobile health application [80], digital literacy [38], and health literacy [82,83].

Implications and future directions
The mixed-methods evaluation conducted in this study and the results represent the first step in optimizing the development and evaluation of Tanya Obat application.A previous investigation assessed satisfaction with telehealth application using SUS method and showed improvement from 71.8 to 82.5 following enhancements based on the first round of testing [84].Therefore, SUS score tends to increase when adequate improvements are provided to Tanya Obat application.These results support the importance of incorporating usability studies as part of the digital health intervention design process [85].
Continuous explorations to obtain post-refinement data with a larger sample size from a multicenter study in different provinces in Indonesia are essential to provide more accurate representative results.Additional investigations are required to evaluate the impact of Tanya Obat application, particularly among patients with chronic diseases who regularly consume medications, as usability may vary between different clinical groups.This can help identify context-related issues in the future, such as patient adherence, access to healthcare, and satisfaction.

Strengths and limitations
The strengths of this study include the recruitment of two different subjects, namely the general population and pharmacists, representing potential and relevant end users.Furthermore, the quantitative and qualitative data provided by real-world users are crucial in ensuring the addition of obtained views in the application development to improve usability.The inclusion of real-world participants showed important usability problems and unidentified solutions during the development or expert panel review stages.However, one main limitation of convenience sampling is the risk of bias due to the lack of random selection.Certain groups within the population may be overrepresented or underrepresented due to the inability to control the questionnaire link distribution.This skew might lead to the production of results not accurately representing the entire population.For example, the majority of participants were aged below 50 years and had a moderate to high educational level, both of which were factors increasing the tendency of technological proficiency and willingness to adopt new technology [86,87].In future investigations, combination of multiple methods, such as random, stratified, or systematic sampling, can improve the quality and representativeness of the convenience sample, which tends to produce more accurate and reliable results, thereby enhancing generalizability.Furthermore, the subjectivity of respondents may be resulting in under or overestimating their experience in using smart-phones which may lead to the non-significant association with the usability score.There was a challenge in ensuring full exploration of all Tanya Obat application features, and the reliance on screenshots from feedback provided by participants without direct verification constituted potential limitations.Future investigations need to conduct a real-time observation or the recording of screen activity as a more objective measurement [88].The method of this study was limited to user-based usability evaluation, focusing solely on the satisfaction aspect, and not incorporating the think-aloud method [89].Additionally, expert usability evaluation through heuristic tests [90] were not performed, generating a less comprehensive scope for the results obtained.Despite these limitations, Tanya Obat application usability was improved before launching by making changes based on the concerns and recommendations of the target user groups.

Conclusion
In conclusion, the results showed that the developed Tanya Obat application was below average in perceived usability.Therefore, specific feature optimizations should be performed, particularly in terms of visual appeal, features, and system functionality, to improve potential acceptance and usability, facilitating successful adoption in Indonesia.

"
It was bothering me because of the inconsistency of font style and color.Please fix this issue." (Participant 11) 2 Features Improvements 22 (22) Expand the network of partnered pharmacies and pharmacists to cover regions effectively."It seems that only a small number of pharmacies and pharmacists have agreed to be partner in this application, but the number of pharmacies and pharmacists need to be increased." (Participant 23) Locate nearby pharmacies 15 (15) Provide additional information about pharmacies, such as opening and closing hours."I want to know more information about pharmacies, such their opening and closing hours." (Participant 17) Enhance the filter options with selections for city, neighborhood, district, opening hours, and ratings.

Table 1
The characteristics of participants

Table 2
Associations between the characteristics of the general population and perceived usability (N = 100)

Table 3
Associations between the characteristics of pharmacists and perceived usability (N = 76) Since now the application is only available in Play Store, the development of the app as the iOS version need to be hastened to enhance visibility among users." (Participant 75) "

Table 5
Concerns and recommendations of pharmacists for Tanya Obat application (N = 76)